Virtual grouping of memory

ABSTRACT

The present disclosure includes identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determining a memory segment size from the capacities of the memory modules; identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

TECHNICAL FIELD

This disclosure relates to virtual memory grouping. In particular, itrelates to virtual memory grouping across memory channels.

BACKGROUND

Main memory in a system can be setup in various physical configurationsto optimize power consumption and performance. Memory may be mirroredfor redundancy to mitigate DIMM failures. Fully mirrored memory, whilebeing fully redundant, would cut the capacity of the memory in half vs.non-mirrored memory. However, memory may be selectively mirrored so thata portion of the memory is mirrored, such as that containing informationcritical to program operation, and the remainder of the memory isnon-mirrored.

SUMMARY

The present disclosure includes a method, comprising: identifying, in amemory system, a capacity for each of a plurality of memory modules fora first memory channel having a first amount of memory and a secondmemory channel having a second amount of memory; determining a memorysegment size from the capacities of the memory modules; identifying afirst memory segment of the memory segment size for the first memorychannel and a second memory segment of the memory segment size for thesecond memory channel; and creating a virtual group that includes thefirst memory segment and the second memory segment and that uses lessthan the entire first amount of memory from the first memory channel.

The present disclosure further includes a method, comprising identifyinga capacity for each of a plurality of memory modules for a first memorychannel and a second memory channel in a memory system; determining afirst portion of memory modules and a second portion of memory modulesfrom the capacities of the memory modules for each of the first andsecond memory channels; creating a first virtual group that includes thefirst portion of the memory modules from the first memory channel andthe first portion of memory modules from the second memory channel; andcreating a second virtual group that includes the second portion of thememory modules from the first memory channel and the second portion ofmemory modules from the second memory channel.

The present disclosure further includes a computer system, comprisingservice processor firmware configured to identify a capacity for each ofa plurality of memory modules for a first memory channel having a firstamount of memory and a second memory channel having a second amount ofmemory; determine a memory segment size from the capacities of thememory modules; identify a first memory segment of the memory segmentsize for the first memory channel and a second memory segment of thememory segment size for the second memory channel; and create a virtualgroup that includes first memory segment and the second memory segmentand that uses less than the entire first amount of memory from the firstmemory channel.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings included in the present application are incorporated into,and form part of, the specification. They illustrate embodiments of thepresent invention and, along with the description, serve to explain theprinciples of the invention. The drawings are only illustrative oftypical embodiments of the invention and do not limit the invention.

FIG. 1 is a flowchart of a method for creating virtual groups of memory,according to embodiments of the disclosure.

FIG. 2A is a diagram of a memory system, according to embodiments of thedisclosure.

FIG. 2B is a diagram of memory allocation in a memory system, accordingto embodiments of the disclosure.

FIG. 3 is an example diagram of a memory system, according toembodiments of the disclosure.

FIG. 4 is an example diagram of grouping of memory segments for thesystem of FIG. 3, according to embodiments of the disclosure.

FIG. 5A is a diagram of memory from memory controller segments inphysical groups for physical grouping, according to embodiments of thedisclosure.

FIG. 5B is a diagram of memory from memory controller segments invirtual groups for virtual grouping 1, according to embodiments of thedisclosure.

FIG. 5C is a diagram of memory from memory controller segments invirtual groups for virtual grouping 2, according to embodiments of thedisclosure.

DETAILED DESCRIPTION

Memory modules in a memory system may be organized into groups alongmemory channels behind memory controllers and memory buffers to matchthe internal throughput of the system bus of a computer system. Forexample, a memory controller may have multiple memory buffers, eachmemory buffer having multiple memory channels leading to one or moreDIMMs. Memory modules may be populated into memory channels in groups ofsimilar sizes of memory to enable memory minoring and memoryinterleaving, while unequal sizes of memory may be populated to formdifferent groups. An operating system or hypervisor may allocate memoryfor an application as a physical group across multiple memory modules sothat those memory modules may be accessed at the same time. The memorymap for a particular physical group will likely be a single contiguousspace.

Accessing memory according to the physical organization of the memorybehind the memory controller may compromise memory throughput andaffinity. Access to the memory groups may be limited by the physicalcharacteristics of the memory, as the grouping is performed alongphysical boundaries. Mirrored data may be allocated to physical memorygroups of equal sizes behind a memory buffer, which may reduce thememory flexibility and utilization of the system. Memory groupedaccording to a particular bandwidth may not be optimized for the way inwhich the memory is actually utilized by applications.

According to embodiments of the disclosure, memory may be virtuallygrouped and accessed to improve memory throughput and affinity. Memorysegments of an identified memory size may be grouped virtually accordingto considerations such as the minimum memory module capacity,application priority, criticality, or memory access characteristics, andperformance data for various memory module configurations. Virtualgroups accessed by multiple memory controllers may scale the throughputhigh, as a greater number of memory modules may be accessed in parallel.Virtual groups of equal memory segments may be created from the samephysical memory group or from physical memory groups of different sizesand behind multiple memory controllers. Memory presented as andallocated through virtual groups may be accessed symmetrically anddiffusely through operations such as memory interleaving.

Performance and configuration information collected from memory in thevirtual groups may be utilized by systems and users to improve physicalmemory access and application performance. The performance of memory invarious virtual configurations may be used to create improved physicalgroupings of the memory modules through memory configuration andpopulation rules. Data may be allocated to a particular memory groupbased on the priority of the application's data or the memory accesscharacteristics of the application. Virtual groups may also supportselective memory mirroring by identifying data to be mirrored andconfiguring virtual memory groups accordingly.

Method

FIG. 1 is a flowchart of a method for creating virtual groups of memory,according to embodiments of the disclosure. The memory capacities of oneor more memory modules in the memory system may be determined, as in100. Memory from the memory modules may be grouped into virtual groups,as in 110. Once the virtual groups are created, memory from the virtualgroups may be allocated to applications, as in 120. The memory moduleshaving memory in a virtual memory group may be monitored forperformance, power consumption, and other usage statistics, as in 130.

To determine memory capacities of the memory modules, as in 100, thesystem may perform an initial program load, as in 101. Once the systemhas been initialized, vital product data for the memory modules may becollected, as in 102. Vital product data may include hardwareinformation, such as serial number and capacity. The vital product datamay be determined for the system or collected from a vital product datastorage location on the system. The collection of vital product data maybe performed by a system such as the host system of the memory system, aremote system, or a field replaceable unit.

Memory from the memory modules may be grouped into virtual memorygroups, as in 110. The capacities of the memory modules may be scannedand evaluated from the vital product data of the memory modules, as in111. A memory segment size (or portion) may be determined from thescanned memory module capacities, as in 112, according to the memoryaccess characteristics of the memory module. In certain embodiments, thememory segment size may be the smallest independently operated memoryunit for one or more physical memory groups, such as the smallest memorymodule size or a rank size in a memory module. The memory segment sizemay be evaluated for memory modules across two or more memory channels.For example, if a first memory channel has two 1 GB DIMMs, a secondmemory channel has four 1 GB DIMMs, the memory segment size may be thesmallest memory module size in common between the two memory channels.In this example, 1 GB may be the memory segment size; alternately, 2 GBmay be the memory segment size, as both groups have at least two 1 GBDIMMs.

Memory segments of the memory segment size may be identified for memorymodules in the memory system, as in 113, and used to create a virtualmemory group, as in 114. In the example above, the two 1 GB DIMMs fromthe first channel and two 1 GB DIMMs from the second channel may begrouped into a first group. If a third memory channel has two 1 GBDIMMs, the remaining two 1 GB DIMMs from the second memory channel maybe grouped with the two 1 GB DIMMs from the third memory channel.

The virtual groups may contain memory from memory channels that is lessthan the total memory capacity of the memory channels. In someembodiments, a memory buffer or controller having two memory channelsmay have memory grouped into two or more virtual groups, each groupcontaining memory from both memory channels. In other embodiments, avirtual group may have memory from memory channels of different memorycapacity connected to two different controllers. In other embodiments, avirtual group may have memory from memory channels having differentmemory capacities.

Creation of virtual groups may be informed by performance information ofthe memory modules, as in 131. For example, if an executing processor ina distributed shared memory system has greater processor affinity to acertain group of DIMMs, those DIMMs may be used for the virtual groupassigned to an application executed on that processor.

Once the virtual groups are created, memory from the virtual groups maybe allocated to applications, as in 120. The application's priority,criticality, or memory access characteristics may inform the operatingsystem or hypervisor as to which virtual group may be allocated to theapplication, as in 121. For example, if an application is known to havefairly linear memory access or is non-critical, a virtual groupproviding deeper memory access or limited to fewer memory modules may beused, so that memory for applications with more diffuse memory access orrequiring greater fault tolerance may use virtual groups having agreater usage of memory modules. The virtual group may be targeted by orallocated to the application, as in 122. The application's performancemay be monitored to determine the appropriateness of the virtual groupfor that application.

The memory modules in a virtual group may be monitored for performanceand power consumption, as in 130. Information of the memory modules,such as performance and power consumption, may be collected, as in 131.The information may be used as feedback to determine memory moduleconfiguration options, as in 132. These memory module configurationoptions may be used to physically configure or populate the memorymodules so as to be improve memory access, such as by increasing memoryaffinity or optimizing throughput. These configuration options may beused as DIMM plug rules when the DIMMs are off-line.

System

FIG. 2A is a diagram of a memory system, according to embodiments of thedisclosure. A processor 201 may be connected to one or more memorycontrollers 202. The memory controller 202 may be part of the processoritself, as shown in FIG. 2, or on a separate chip. In this embodiment,the processor contains a memory controller A 202A and a memorycontroller B 202B.

Each memory controller may support one or more memory buffers 203. Thememory buffer 203 may be part of the memory controller 202 or on aseparate chip. Shown here, memory controller A 202A is connected tomemory buffer A 203A, and memory controller B 202B is connected tomemory buffer B 203B. Each memory controller 202 or memory buffer 203may be connected to one or more memory modules 204 through one or morememory channels 209. Shown here, memory buffer A 203A is connected tomemory module A 204A and memory module C 204C through memory channel A209A, and memory buffer B 203B is connected to DIMM B 204B throughmemory channel B. The performance of the memory modules 204 and theprocessor 201 may be monitored by performance monitor logic 207.

The memory system may contain a service processor 205. The serviceprocessor 205 may be a hardware or firmware unit configured for virtualmemory management. The service processor 205 may be configured to createvirtual groups 206 of memory from the memory modules 204 of differentmemory channels 209. The service processor 205 may receive feedback fromthe performance monitor logic 207 as to performance of memory modules204 in the virtual groups 206 and applications assigned to certainvirtual groups 206. This performance information may be used by ahypervisor 208 to assign applications to virtual groups 206 or toprovide configuration options for physical configuration of the DIMMs204.

FIG. 2B is a diagram of memory allocation in a memory system, accordingto embodiments of the disclosure. Service processor firmware 214 maycreate virtual memory groups from memory 215. The memory 215 may includememory modules from across a memory system, including memory modulesfrom different physical groups behind different processors, memorycontrollers, and memory buffers. An operating system 212, hypervisor213, or application 211 may allocate memory 215 from the memory groupsto the application 211. The service processor firmware 214 may presentthe memory 215 to the hypervisor 213 or operating system 212 as virtualgroups.

Example

FIG. 3 is an example diagram of a memory system for use in the examplesdiagrams of FIGS. 4, 5A, 5B, and 5C, according to embodiments of thedisclosure. In this example, a processor 300 has eight memorycontrollers, each connected to multiple DIMMs through one or more memorychannels (and optionally one or more memory buffers, not shown). Thememory contained behind each memory controller may be referred to as thememory controller segment (MCS) for the memory controller. Memorycontroller 1 301 controls DIMM 1A 311A and DIMM 1B 311B as MCS 1; memorycontroller 2 302 controls DIMM 2A 312A and DIMM 2B 312B as MCS 2; memorycontroller 3 303 controls DIMM 3A 313A and DIMM 3B 313B as MCS 3; memorycontroller 4 304 controls DIMM 4A 314A and DIMM 4B 314B as MCS 4; memorycontroller 5 305 controls DIMM 5A 315A, DIMM 5B 315B, DIMM 5D 315C, andDIMM 5D 315D as MCS 5; memory controller 6 306 controls DIMM 6A 316A,DIMM 6B 316B, DIMM 6C 316C, and DIMM 6D 316D as MCS 6; memory controller7 307 controls DIMM 7A 317A, DIMM 7B 317B, DIMM 7C 317C, DIMM 7D 317D,DIMM 7E 317E, DIMM 7F 317F, DIMM 7G 317G, and DIMM 7H 317H as MCS 7; andmemory controller 8 308 controls DIMM 8A 318A, DIMM 8B 318B, DIMM 8C318C, DIMM 8D 318D, DIMM 8E 318E, DIMM 8F 318F, DIMM 8G 318G, and DIMM8H 318H as MCS 8. For this example, each DIMM may be equal to 1 gigabyte(GB) of memory for simplicity of explanation.

FIG. 4 is an example diagram of physical and virtual groupings of memorysegments for the system of FIG. 3, according to embodiments of thedisclosure. For a physical grouping, memory from a memory controllersegment (MCS) may be grouped and accessed with memory from other memorycontroller segments having the same size and configuration to formphysical groups for memory access. In this example, three physicalgroups may be formed. Group 1 may contain 8 GB of total memory,comprising the 2 GB of memory behind each of the four memory controllers1-4. Group 2 may contain 8 GB of total memory, comprising the 4 GB ofmemory behind memory controllers 5 and 6. Group 3 may contain 16 GB oftotal memory behind memory controllers 7 and 8.

When the memory is grouped virtually, memory access may not beconstrained to physical memory groups or memory controller segments, anda greater number of memory access configurations may be created. Byvirtually grouping smaller, equal segments of memory across memorychannels, a greater number of memory buffers and controllers may accessmemory at a time, reducing access time to the memory. In this example,virtual groups having the same size as the physical groups discussedabove may be formed, but with different memory access configurations.

For virtual grouping 1, 2 GB of memory from each memory controller 1-8may be grouped as memory segments into group 3 to form a 16 GB group; 2GB of memory from each memory controller 5-8 may be grouped into group 2to form an 8 GB group; and 4 GB of memory from each memory controller 7and 8 may be grouped into group 1 to form an 8 GB group.

For virtual grouping 2, 2 GB of DIMMs from each memory controller 1-4may be grouped as memory segments into group 1 to form an 8 GB group; 4GB of memory from each memory controller 7 and 8 may be grouped intogroup 2 to form an 8 GB group; and 4 GB of memory from each memorycontroller 5-8 may be grouped into group 3 to form a 16 GB group.

FIG. 5A, FIG. 5B, and FIG. 5C are diagrams of the physical and virtualgroupings of memory from FIG. 4, according to embodiments of thedisclosure. FIG. 5A is a diagram of memory from memory controllersegments in physical groups for the physical grouping, according toembodiments of the disclosure. As discussed above, each physical groupcontains memory from memory controller segments of the same size andconfiguration.

FIG. 5B is a diagram of memory from memory controller segments invirtual groups for the virtual grouping 1, according to embodiments ofthe disclosure. Memory access to group 3 is spread across 8 memorycontrollers, which may allow for greater throughput. Processoroperations that require accessing multiple segments of informationsimultaneously, such as parallel processing, or containing critical datamay best utilize the memory access characteristics of virtual group 3.Critical process operations requiring similar fast access or highavailability but for a smaller 8 GB capacity may best utilize the memoryaccess characteristics of virtual group 2. Non-critical applications orapplication that may better utilize access to the same DIMM forinformation, such as deep or linear computing operations, may utilizevirtual group 1. The memory segments from each memory controller segmentare similarly sized, and may be used for symmetrical processes such asmemory mirroring and interleaving.

For memory access administered with a lower overhead or for fewercritical or parallel access operations, a different virtual groupingconfiguration may be used. FIG. 5C is a diagram of memory from memorycontroller segments in virtual groups for the virtual grouping 2,according to embodiments of the disclosure. Applications utilizingfaster memory access for parallel processes or performing more criticaloperations may be assigned group 1. Applications involving non-criticalstorage may be assigned group 2. Operations involving linear memoryaccess or minoring, such as database storage, may be assigned group 3,as access is spread across memory controllers. The 16 GB group capacityof group 3 is divided among four memory controllers for redundancy, twoof which may contain original data and two of which may contain copydata.

Although the present disclosure has been described in terms of specificembodiments, it is anticipated that alterations and modificationsthereof will become apparent to those skilled in the art. Therefore, itis intended that the following claims be interpreted as covering allsuch alterations and modifications as fall within the true spirit andscope of the disclosure.

1. A method, comprising: identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determining a memory segment size from the capacities of the memory modules; identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.
 2. The method of claim 1, wherein the first amount of memory and the second amount of memory are not equal.
 3. The method of claim 1, wherein the first memory channel is connected to a first memory controller and the second memory channel is connected to a second memory controller.
 4. The method of claim 1, wherein the memory segment size is a capacity of one or more memory modules in common for each of the first memory channel and the second memory channel.
 5. The method of claim 4, wherein the memory segment size is a size of a smallest memory module in common for each of the first memory channel and the second memory channel.
 6. The method of claim 1, wherein identifying the capacity for each of a plurality of memory modules comprises: collecting vital product data for each of the plurality of memory modules; and determining the capacity from the vital product data for each of the plurality of memory modules.
 7. The method of claim 1, further comprising: determining memory access characteristics of an application; and assigning the virtual group to the application based on the memory access characteristics.
 8. The method of claim 1, further comprising: collecting performance information from the memory modules having memory in the virtual group; and determining memory module population rules from the performance information for the memory modules having memory in the virtual group.
 9. The method of claim 8, further comprising configuring memory modules in the memory system according to the memory module population rules.
 10. The method of claim 7, wherein at least one memory access characteristic involves memory mirroring.
 11. A method, comprising: identifying a capacity for each of a plurality of memory modules for a first memory channel and a second memory channel in a memory system; determining a first portion of memory modules and a second portion of memory modules from the capacities of the memory modules for each of the first and second memory channels; creating a first virtual group that includes the first portion of the memory modules from the first memory channel and the first portion of memory modules from the second memory channel; and creating a second virtual group that includes the second portion of the memory modules from the first memory channel and the second portion of memory modules from the second memory channel.
 12. The method of claim 11, wherein the first memory channel and the second memory channel are connected to a memory buffer.
 13. The method of claim 11, wherein identifying the capacity for each of a plurality of memory modules comprises: collecting vital product data for each of the plurality of memory modules; and determining the capacity from the vital product data for each of the plurality of memory modules.
 14. The method of claim 11, further comprising: determining memory access characteristics of an application; and assigning the virtual group to the application based on the memory access characteristics.
 15. The method of claim 11, further comprising: collecting performance information from the memory modules in the virtual group; and determining memory population rules from the performance information for the memory modules in the virtual group.
 16. The method of claim 11, further comprising configuring memory modules in the memory system according to the memory population rules. 17-20. (canceled) 